Add optional streaming feature using Chat API #55
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Update package.json Update aws packages to v3.620.0 Fix event streaming bug with ChatAPI. Bugfix here: https://github.com/aws/aws-sdk-js-v3/releases/tag/v3.620.0 Prereq for Chat API feature. Update IAM permissions for Chat API Add Chat to IAM role. Prereq for Chat API streaming feature. Add slack-stream-event lambda handler To allow the optional streaming feature of ChatAPI we add an additional lambda handler. Admin can choose between which endpoint will handle slack events. Modify helper functions to fit both ChatSync & Chat Keep code DRY by not remaking each helper function. 1. Add "commandType" to chat & callClient to determine return type. 2. Modify getResponseAsBlocks inputs to fit Chat API outputstream. 3. Add optional MetadataEvent typing to saveMessageMetadata & getFeedbackBlocks for ChatAPI. Remove `chat` function and directly use client calls 1. Remove chat function to avoid double conditionals 2. Add the sendChatSyncCommand & sendChatCommand functions to dependencies. Add updated unit tests for ChatSync & Chat 1. Add mock responses for Chat API 2. Modify mocks to have two separate calls for ChatSync/Chat. 3. Modify unit tests to fit (2). 4. Create new unit test file for slack-stream-event-handler.ts. All the tests are the same as slack-event-handler.test.ts except for the 5 that start with "Should chat" -> Modified to fit Chat API output. refactor: remove 'any' type from input object in sendChatSyncCommand Update slack event handler to use modified helpers Create slack-stream-event-handler 1. Optional event subscription endpoint for slack. 2. Code is identical to slack-event-handler.ts until line 334. Update READMEs with chat stream feature setup Remove unnecessary variable `latestTextEvent` Previous implementations required the latestTextEvent for future references after loop -> this is no longer required. Update CHANGELOG date Modify `sendChatSyncCommand` and `sendChatCommand` to be thin wrappers 1. Moved the try catch & truncating to amazon-q-helpers.ts & dumbed logic down in amazon-q-client.ts functions to only be thing wrappers. Modify functions to fit `callChatSyncCommand` & `callChatCommand`
Add Lambda Handler for Slack-Gateway streaming & update helper functions for Chat API.
leenmin-aws
changed the title
Add bufferSize config variable
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Q-Slack-Gateway uses ChatSync which results in an unoptimal customer experience.
New feature provides an additional lambda function that leverages Chat API to stream LLM response in real time.
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.